Industry Watch: Is 2007 the Year of Question-Answering?

Back at the beginning of 2000, I was a member of a working group tasked to come up with some guidelines for revamping my University’s website. During one of our meetings, someone made the suggestion that decisions about how to structure and present information on the website should be driven by the kinds of questions that users come to the site with. Suddenly a light went on, and there appeared an idea for data gathering that might provide us with some useful information. To find out what people really wanted to know when they visited the website, we would replace the University’s search engine by a page that invited the user to type in his or her query as a full natural language question. Appropriately chosen examples would be given to demonstrate that using real questions delivered better pages as a result. The data gathered would tell us what people were really looking for, more than could be gleaned from conventional search queries, and would therefore help us to better structure the information available on the website. Thanks to a very cooperative University webmaster and a supportive administration, our JustAsk! web page was up and running within weeks. Of course, we didn’t actually implement a question-answering system. Instead, behind the page was a script which, given any submitted query, would strip out any stop words it contained (after logging the full query, of course), and then simply pass the remaining words in the query on to the same search engine that had been used previously. Within a few days of the deception being put in place, a number of people independently commented on how much better the University’s search engine was, now that we had added a natural language capability. I came clean and denied all responsibility for any perceived improvements, of course; but it’s entirely possible that users did get better results simply because, even after stripping out stop words, their queries tended to become longer, and contained more content words, when expressed as questions. Every Sunday at midnight since then, I have been sent a log file containing the previous week’s queries. The web page in question has now moved and become